# Adaptive softmax

Transformer-XL is a causal Transformer architecture that uses relative position encoding. It can capture longer context by reusing previously computed hidden states, making it suitable for text generation tasks.